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Abstract 

Motivated by the growing evidence of the importance of charge fluctuations in the pseudogap 
phase in high-temperature cuprate superconductors, we apply a large- iV expansion formulated in 
a path integral representation of the two-dimensional t-J model on a square lattice. We study all 
possible charge instabilities of the paramagnetic state in leading order of the 1/N expansion. While 
the d-wave charge density wave (flux phase) becomes the leading instability for various choices 
of model parameters, we find that a d-wave Pomeranchuk (electronic nematic phase) instability 
occurs as a next leading one. In particular, the nematic state has a strong tendency to become 
inhomogeneous. In the presence of a large second nearest-neighbor hopping integral, the flux 
phase is suppressed and the electronic nematic instability becomes leading in a high doping region. 
Besides these two major instabilities, bond-order phases occur as weaker instabilities close to half- 
filling. Phase separation is also detected in a finite temperature region near half-filling. 

PACS numbers: 71.10.Hf, 74.72. Gh, 74.72.Kf 
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I. INTRODUCTION 



The pseudogap (PG) phase in cuprate superconductors provides one of the most ac- 
tive subjects on high-T c superconductivity. The PG phase is characterized by highly 
anomalous properties^ 2 - which are rather universal for all cuprate superconductors. One of 
the puzzling observations comes from angle-resolved photoemission spectroscopy (ARPES) 
measurements,- which revealed arc-shaped disconnected Fermi surfaces,- called Fermi arcs, 
instead of a large Fermi surface. In underdoped cuprates the PG opens below a temperature 
T*, which is far above the superconducting transition temperature T sc . Furthermore, in con- 
trast to the behavior of T sc , T* increases with decreasing doping in the underdoped region. 
The PG is very anisotropic along the Fermi surface. It has a maximal gap in the (0, 0)-(0, it) 
direction (antinodal direction) and vanishes upon approaching the Brillouin zone diagonal 
(nodal direction), similar to the <i-wave superconducting gap. 

In spite of the consensus on the existence of the PG, its origin and nature remain elusive. 
There are two major scenarios. One is that the PG originates from preformed pairs above 
T sc $&- The other is that the PG is distinct from the superconducting gap and associated with 
a certain order which competes with superconductivity, but both coexist at low tempera- 
ture, leading to "two gaps" in the electronic spectrum.-"- Several phenomenological models 
which are in favor of the two-gap scenario were already studied in various contexts, but in- 
voking different orders, such as d-wave charge density wave (dCDW), 1 — <i-wave Fermi surface 
deformations, — charge density wave^— including stripes,—^ phase separation (PS),— 
and others such as resonating-valence-bond-type charge order™ and loop-current order.— 

The ciCDW is a flux phase, where orbital currents flow around each plaquette in a stag- 
gered pattern. The electronic spectrum in the flux phase has a gap with ci-wave sym- 
metry, the same as the superconducting gap symmetry. The flux phase was obtained in 
the large- N approach to the t-J model in various formalisms 2 ^— and the presence of flux 
correlations was confirmed by the exact diagonalization.— On the other hand, in the Hub- 
bard model, the dynamical cluster approximation failed to detect static long-range order 
of the dCDW— whereas the variational cluster approximation showed that the <iCDW is a 
metastable solution. 28 Fluctuations associated with the <iCDW can provide a route to ad- 
dress the PG. A perturbative analysis of the electron self-energy due to dCDW fluctuations 
catches many important features observed by ARPES, not only a PG and its associated 
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Fermi arcs^ 3 ^ but also a semiquantitative aspect of renormalization of the electron band 
dispersion in the PG phase.— 

The (i-wave Fermi surface deformations are driven by a <i-wave Pomeranchuk— instability 
(dPY), leading to an electronic nematic state.— In this state, an orientational symmetry of the 
systems is broken without breaking however translational invariance. The <iPI was found in 
the slave-boson mean-field,— exact diagonalization,— and variational Monte Carle 36 studies 
in the t-J model, and also in the Hubbard model.— r — The dPl itself does not become the 
leading instability in most of theoretical studies. However, it was pointed out that the models 
retain appreciable correlations of the dPl,— ^ which then may lead to a giant response 
to a small xy anisotropy. Such a giant response was actually observed in the PG region 
in YBa2Cu30^, which has a small anisotropy originating from the orthorhombic crystal 
structure, by neutron scattering 4 ^ 4 ^ and transport measurements.— Theoretical studies for 
the former— ^ and the latter— confirmed that idea. 

Charge-stripe order is extensively discussed for cuprates.— ^ Since the charge-stripe order 
breaks both orientational and translational symmetry of the system, the stripe phase is 
also called an electronic smectic phase 3 - 3 - and has lower symmetry than the nematic phase. 
The experimental observation of charge order in La-based cuprates^ 6 - provides grounds to 
consider the stripe order. A charge-stripe solution was indeed obtained in the density matrix 
renormalization group (DMRG) study 4714 - in the t-J model. However in the presence of the 
second nearest-neighbor hopping integral the charge-stripe order turned out to be unstable 
in the t-J model.— ^ 

PS is also another possible instability in the t-J model.— It is however still highly 
debated whether the model indeed shows the instability toward PS^^ 4 - or not^^^— in a 
parameter region realistic to cuprates. Although PS is in general suppressed by long-range 
Coulomb forces, strong charge fluctuations in the proximity to PS can be important and 
responsible for anomalous properties in the PG phase and superconductivity— In fact, the 
proximity to PS plays an important role to generate a singular interaction between electrons 
at zero momentum transfer as shown in the infinite-?/ Hubbard Holstein model.— ^ When 
long-range Coulomb interactions are added, the singularity shifts to a finite momentum 
transfer, leading to an incommensurate charge density wave similar to stripes.— 'i 3 - 

Theoretically it is believed that the two-dimensional (2D) t-J and Hubbard models con- 
tain the main ingredients for describing cuprates,— i.e., antiferromagnetism at zero doping, 



3 



a metallic state at finite doping, and a strong tendency to <i-wave superconductivity. Given 
that various charge instabilities are invoked to address the PG, and also other anomalous 
properties in cuprates, it is interesting to study what kind of charge instabilities are favored 
in the 2D t-J model by treating all possibilities on equal footing in a controllable scheme. 

In this paper, we analyze the 2D t-J model in terms of Hubbard operators by including 
the nearest-neighbor Coulomb interaction V to avoid a subtle feature of PS; our main results 
are not affected by the presence of V. We apply a large- N expansion formulated in a path 
integral representation.— >2ii£2 In this approach the two spin components are extended to iV 
and an expansion in powers of the small parameter 1/N is performed, providing a controllable 
scheme without a perturbative expansion in any model parameter. In addition, different kind 
of instabilities can be studied on equal footing, allowing us to perform a stability analysis on 
all possible charge instabilities already at leading order. We find that the t-J model shows 
tendencies to the flux and electronic nematic state in a wide doping region. In particular, 
the nematic state has a strong tendency to become inhomogeneous. Close to half-filling, 
bond-order phase (BOP) and PS are also obtained. 

In the next section, we first provide a brief summary of our theoretical scheme and then 
explain the most important charge instabilities, <iCDW, dPl, BOP, and PS. Our results are 
presented in Sec. Ill and are discussed by comparing with literature in Sec. IV. Implications 
for cuprates are also discussed in the same section. 

II. THEORETICAL FRAMEWORK 

A. Large- AT approach to the t-J-V model 

In a previous paper,— a large- N expansion for the t-J-V model was formulated in terms of 
a path integral representation for the Hubbard X-operators. For the sake of a self-contained 
presentation, we first summarize the formalism. 

The t-J-V model is described by the following Hamiltonian, 

H = -"J*] tijc\ a c ja + J ^(Si ■ Sj - ^UiTij) + V^2 n i n i > (!) 

where tij = t (t') is the hopping integral between the first (second) nearest-neighbor sites 
on a square lattice; J and V are the exchange interaction and the Coulomb repulsion, 



4 



respectively, between the nearest-neighbor sites. The main role of the V^-term in the present 
study is to suppress the tendency toward PS while, in other works,— ^ the V^-term was 
included to investigate its effect on superconductivity. c\ a and c ia are the creation and 
annihilation operators of electrons with spin a (a =4,t), respectively, under the constraint 
that the double occupancy of electrons is excluded at any site i. rij is the electron density 
operator. 

The electron and spin operators are connected to Hubbard operators 65 via c] a = X?°, 
c ia = X^, S+ = Xj\ = Xf, and m = X^ + Xf 1 . The operators Xf and X° CT are 
called fermionlike, whereas the operators X[ a ' and X?° are called bosonlike; X®° will be 
introduced later [Eq. 01])]. After writing Hamiltonian (1) in terms of the Hubbard operators, 
we extend the spin degree of freedom to N channels and obtain the Hamiltonian in the 
large- iV formalism, 

hJ,P (i,j),PP' {i,j),PP' hP 

(2) 

The spin index a is extended to a new index p, which runs from 1 to N. In order to obtain a 
finite theory in the iV-infinite limit, t, t', J and V are rescaled as t/N, t'/N, J/N and V/N, 
respectively. The chemical potential /i is introduced in Eq. ([2]). 
In the path integral formulation our Euclidean Lagrangian reads 



T i^ (xrxr + xf xt) 

L e = g 1^ ^oo + u ( d ) 

i,p 1 

with the following two additional constraints, 

^ 00 +$>r-f = o, (4) 



and 



XP°x 0p ' 



x r - -^t- = o > (5) 

which are imposed on the path integral via two 5-functions. In Eq. ([3]), X^ = d T Xi p0 
and r is the euclidean time, namely r = it. Equation (jJJ) is the iV-extended completeness 
condition. The form of the kinetic term in the Lagrangian Le, as well as the constraint 
Eq. comes from the requirement that the X-operators should fulfill their commutation 



rules. For details we refer to Ref. |61|. In the path integral approach we associate Grassmann 
and usual bosonic variables with fermionlike and bosonlike X-operators, respectively. 

We now discuss the main steps needed to introduce a large- iV expansion.— £2 First the 
V-term in the Hamiltonian is written in terms of by using Eq. (]3J) . We then eliminate 
the bosonic variables X pp ' by implementing the <5-function associated with Eq. (J5J). The 
completeness condition [Eq. P|] is imposed by introducing Lagrange multipliers Aj. We 
write X 60 and Aj in terms of static mean-field values, r and Ao, and fluctuation fields, 5Ri 
and 5\i, 

X9° = Nr (l + SRi) 
A, = A + 5A 4 . (6) 

In addition, we introduce the following fermion fields 66 defined by 

ft = 1 v p0 

f - = vk x ° P - (7) 

The exchange interaction is then described by four fermion fields, which are decoupled 
through a Hubbard- Stratonovich transformation by introducing a field associated with a 
bond variable, 



A jy W (8 ) 

The field Ay is parameterized by 

A? = A(l + r? + iA ? ), (9) 

where and A% correspond to the real and imaginary parts of the fluctuations of the bond 
variable, respectively, and A is a static mean-field value. The index rj takes two values 
associated with the bond directions rji = (1,0) and r] 2 = (0,1) on a square lattice. After 
expanding 1/(1 + SR) in powers of 5R, we obtain an effective Lagrangian, which can be 
written in terms of a six-component boson field 

SX a = (5R , 5\, r" 1 , r" 2 , A m , A m ) , (10) 

the fermions f p , and their interactions. 
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From the quadratic part for fermions we obtain an electronic propagator in the param 
agnetic phase, 

1 



G(k, iv n ) 



(11) 



Here k and w n are the momentum and fermionic Matsubara frequency, respectively, and the 
electronic dispersion is 



£k = — 2(tr + A) (cos k x + cos k y ) — 4:t'r cos k x cos k y — // . 



(12) 



Here Ao in Eq. fl6]) was absorbed in the chemical potential \i. 

From the completeness condition [Eq. (J3])] r is equal to 5/2, where S is the hole doping 
rate away from half-filling. The field A is given by the expression 

J 



S k,?7 



(13) 



where rip is the Fermi function and N s is the total number of lattice sites. For a given 
doping, /i and A are determined self-consistently by solving Eq. ( TT3|) and 

0- ~ *) = jrT,"^ ' (14) 
s k 

The quadratic part for 5X a defines a 6 x 6 bare bosonic propagator D^ b \c[,iu n ), which 
after Fourier transformation reads, 



( t( v -%) t cos (^) + cos (^)] I \ 


0^000 








0^0 
^ 












4A 2 



(15) 







where q and iu n are the momentum and bosonic Matsubara frequency, respectively. The 
quantity D^(q, ioj n ) describes all possible types of bare charge susceptibilities. The bare 
susceptibilities are renormalized already at leading order to become dressed ones, which are 
given by the Dyson equation 



(16) 



Following the diagrammatic rules in Ref. |25|, the 6x6 boson self-energies H a b are computed 

as 



rr i ■ \ N \^u n xU^k-q) - n F (e k ) . 

Il a6 (q, \U) n ) = -— ) h a (k, q, £ k - £ k - q ) — ■ h b (k, q, e k - e k _ q ) 

l\ s lLO n — £ k + £ k _ q 

s k 

The prefactor N in front of the right hand side of Eq. (17) comes from the sum over the 
N channels of p. Thus, the 6x6 boson self-energies n a & are of the same order as [D? ] _1 
[see Eq. (15)]. In Eq. (17) h a is an effective six-component interaction vertex which comes 
from the interaction terms between bosonic and fermionic fields derived from the effective 
Lagrangian. The explicit expression for h a is given by 



;i; 



h a (k, q, v) = i 2£k q + ^ + 2/i + 2A cos (k x - cos (^j + cos (k y - cos ^ 

-2A cos (k x - ^) ; -2A cos (k y - | ) ; 2A sin - ^ ) ; 2A sin (k y - y ) } 

(18) 

From the iV-extended completeness condition [Eq. ([3])] we see that the charge operator 
X 00 is O(N), while the operators X pp are 0(1). Consequently, the 1/N approach emphasizes 
the effective charge interactions. In fact, while in leading order charge susceptibilities contain 
collective effects, they enter the spin susceptibilities in the next-to-leading order. Similarly, 
superconductivity appears in the next-to-leading order.— ^ Therefore, instabilities of the 
paramagnetic phase are expected only, in leading order, in the charge sector. 

In leading order, our formalism agrees with the 1/N slave-boson formalism.™ However, in 
the present approach the fermion variables fi P are proportional to the X-operators [Eq. (J7j)] 
and should not be confused with the spinons in the slave-boson approach. In addition, 
5R [Eq. (JBJ] is proportional to charge fluctuations and not related to holons. Since the 
X-operators are treated as fundamental objects, problems associated with fluctuations of 
gauge fields in the slave-boson approach 67 ' are avoided. Our formalism was also checked 
to yield results consistent with the exact diagonalization 6 ^^ as well as results in another 
formalism of the 1/N expansion in leading order. 24 
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B. Instabilities of the paramagnetic phase 



An instability of the paramagnetic phase is signaled by the divergence of the static sus- 
ceptibilities defined by D ab (q,iu n ) for a continuous phase transition. Therefore we study 
eigenvalues and eigenvectors of the matrix [D ab (c[, iw n )] -1 at iu n = 0. When an eigenvalue 
crosses zero at a given doping rate, temperature T, and momentum q, an instability oc- 
curs toward a phase characterized by the corresponding eigenvector. We have found five 
instabilities associated with eigenvectors V a explained below. 




FIG. 1: Sketch of various phases appearing in our work. Commensurate orders for (a) dCDW, (b) 
dPI, (c) BOP x , and (d) P>OP xy in real space. The commensurate dPI has a momentum q = (0,0) 
whereas the commensurate dCDW, BOP x , and BOP^ have q = (vr,7r). Solid and dashed lines in 
(b)-(d) represent the strong and weak bonds, respectively, (e) Fermi surface deformations (black 
line) associated with the commensurate cfPI; the original Fermi surface is sketched by gray lines. 

a) V a = -^(0,0,0,0,1,-1), which corresponds to the freeze of the imaginary parts of 
the bond variable [Eq. (jHJ)]. The pure imaginary contribution to the hopping term generates 
a net magnetic flux in each plaquette, leading to the instability toward the flux or dCDW 
phase as already found previously.— - — The commensurate flux phase is characterized by the 
modulation vector q = (tt, tt) and describes staggered circulating currents as sketched in 
Fig. [H (a), whereas the incommensurate phase is characterized by q ^ (tt, tt). 

b) V a = ;^g(0, 0, 1, —1, 0, 0), which corresponds to the freeze of the real parts of the bond 
variable. This eigenvector corresponds to the commensurate [q = (0 , 0)] or incommensurate 
[q ^ (0 , 0)] instability toward the dPI phase. The commensurate phase is sketched in Fig. [1] 
(b) in real space. In momentum space it leads to Fermi surface deformations where the 
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Fermi surface expands along the k y direction and shrinks along the k x direction [Fig. [T] (e)], 
or vice versa if the bond along the y direction would become stronger than the x direction. 
While the commensurate dPl has been discussed since 2000,-^2. an incommensurate ciPI 
starts to be discussed very recently.— ~— 

c) V a = (0,0,1,0,0,0) ((0,0,0,1,0,0)), which correspond to the freeze of the third 
(fourth) component and describe the instability toward the BOP x . (BOPy).— - — The corre- 
sponding modulation vector turns out to be q = (it, it) or very close to it. The commensurate 
BOP x , namely with q = (it, tt), is sketched in Fig. [1] (c) whereas the BOP,, with q = (jr, tt) 
is obtained by rotating Fig. [TJ (c) by 90°. 

d) V a = ^=(0,0,1,1,0,0), which corresponds to the freeze of both third and fourth 
components simultaneously. The modulation vector is q = (tt, tt) or very close to it, as in the 
case of the BOP x and BOP y . We refer to this instability as the BOP xy . The commensurate 
BOP xy with q = (tt, tt) is sketched in Fig. [T](d). For simplicity, we also use the phrase BOP 
when we do not have to distinguish between BOP x , BOP y , and BOP^. 

e) V a = (1,0,0,0,0,0), which corresponds to the freeze of charge fluctuations 5R and 
describes the instability toward PS for q = (0, 0) and a charge- density- wave phase, including 
stripes, for a finite q. A finite q instability, however, was not detected in the present study. 

In general, eigenvectors of [D a b(q, ico> n )] -1 can have a non-zero value in each component. 
However we checked that the inner product between an eigenvector of [D a b(q, iwn)] -1 and 
V a becomes larger than 0.99 at the corresponding critical point. 

C. Effective susceptibilities 

While numerical results presented in this paper are computed from the full susceptibility 
Eq. f )16p . it is instructive to extract an effective susceptibility associated with each instability 
explained in the previous section by discarding the interactions with other modes contained 
in D ab (q, iu n ). 

The usual charge-charge correlation function is written as 



Using the completeness condition [Eq. @] and the relation between X®° and 5Ri [Eq. (jSJ)], 




(19) 
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<r 2 



Xfj can be written in Fourier space,— 

X c (q,io; n ) = ~ N [l) D 11 (q,iu n ). (20) 

Thus the charge-charge correlation function is just the component (1, 1) of the D a b. Note 
that the factor N in front of the right hand side of Eq. (|2"0"|) shows that charge fluctuations 
are of 0(1) since D a b oc 1/N as seen in Eq. ffT5j) . 

The susceptibility of the <iCDW is obtained by focusing on the sector a, b = 5, 6 of the 
matrix D~^. We obtain 

XdCDw(q, = [(8/ J) A 2 - n dCDW (q, iu^)]" 1 , (21) 

where n^cDW^ i^n) is the electronic polarizability of the dCDW and is given by 

rr f ■ \ 1 2 / i N^F^k+q) - ^F(fk) /ooA 
n dC Dw(q, Mn) = rr 2_j 7dCDw(q ; k ) ■ , ( 22 ) 

e k+q — e k — iu n 

with a form factor 7dCDw(q> k) = 2A[sin(A;. c + g x /2) — sin(fcj / + f^/2)]. 

Similarly, the susceptibility of the <iPI is obtained from the sector a, b = 3, 4 of the matrix 
D- b l : 

XdPi(q, iw n ) = [(8/ J)A 2 - n dPI (q, i^)]" 1 (23) 

and the electronic polarizability of the <iPI reads 

rr t ■ \ 1 2 / i N^(ek+ q ) - Wp(ek) , .v 
ndpi(q,iw n ) = - — > 7dPi(q> k ) : , (24) 

^ Ck+q — Ck — l^n 

with a form factor 7dpi(q, k) = 2A[cos(A; x + q x /2) — cos(k y + q y /2)]. 
For the case of the BOP x we focus on the sector a = b = 3 and obtain 

Xbop, (q, iwn) = [(4/ J) A 2 - IIbop* (q, iO] -1 , (25) 
where the electronic polarizability is given by 

rr / • \ 1 , A 2 2// , /0 x ^F(ek+q) - ^F(Ck) /ofi x 

nBOP,(q,i^n) = —rr > 4A cos (A^ + gz/2) : . (26) 

^v s ek+q — ek — iu;„ 

For the case of a = b = 4, i.e., BOP y , the form factor in Eq. ( 1261) is replaced by cos 2 (k y +q y /2). 
It is easily seen in Eq. (126]) that the BOP x and BOP,, instabilities occur simultaneously, but 
with a different modulation vector: suppose q = (q x , q y ) for the BOP x ., then q = (q y , q x ) for 
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the BOPy. While we will not present results for the BOP,,, it should be understood that 
the instability of the BOP y also exists. 

The susceptibility associated with BOP xy is given by the same equation as Eq. (|23p . 
except that the form factor 7^pi in Eq. (I24p is replaced by jBOP^y (q 3 k) = 2A[cos(fc a; + 
q x /2) + cos(k y + q y /2)]. 

The form factor 7dCDw(q, k) [7<fpi(q, k)] has a k dependence of cos k x — cos k y at q = (n, ir) 
[q = (0,0)], which indicates the ci-wave character of the instability. Note that the <iPI and 
rfCDW belong to different eigenspace and are not connected with each other by changing 
the momentum q. 

While the terminology of the <iPI itself makes sense when a modulation vector is close 
to q = (0,0), we may consider formally a large q in Eqs. ( l2"B"j) and (|24p . The dPl is then 
connected with the BOP x and BOP y when q is located along the direction of (tt, 0)-(7T, 7r) 
or (0, 7r)-(7T, 71"). Suppose q' = (ir,q y ), we can easily find 

u dP1 (q, iui n ) = n BO p.(q', v*> n ) + n B op y (q', w n ) , (27) 

by noting that 

-L V sin k x cos(k y + q y /2) ^k +q ~ ^k) = . (28 ) 

iV s ^ Ck+q' — Ck ^ 

We thus obtain 

X d M, i^„) = Xiop, + Xbop, (q'« • (29) 

In particular, when q' is equal to Q = (n, ir), Eq. (1291) is reduced to 

XdPi(Q, iw„) = ^Xbop^Q, iw„) , (30) 
because Xbop 2 ,(Q, i^n) = Xbop„(Q, iw„). Similarly, we can also obtain 

XdPl(Q, iw ft ) = Xbop„,(Q, iw n ) • (31) 

Hence when the static BOP susceptibility diverges at q = (tt,tt), the c?PI susceptibility 
also diverges simultaneously at the same momentum unless it already diverges at a different 
momentum. In fact, the dPl with q = (tt, tt) is equivalent to the BOP^ and is interpreted 
as superposition of the BOP x and BOP,, as seen in Fig. [TJ (d). 
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III. RESULTS 



We choose the parameters, J/t = 0.3 and V/t = 0.5. We set t = 1, and all quantities with 
dimension of energy are in units of t. Irrespective of the presence of the V-term, our theory 
catches intrinsic charge instabilities in the 2D t-J model such as c?CDW, dPI, and BOP, which 
are driven by the J-term. We compute the full susceptibility Eq. ( !T6|) for various choices of t' 
by assuming the paramagnetic phase and determine possible charge instabilities in the plane 
of hole density S and temperature T. At half-filling an analytical solution is obtained and the 
c?CDW, dPI, and BOP have the same onset temperature, T c = J/8 = 0.0375t, at which the 
static field A [Eq. (J3])] also sets in. Away from half-filling (5 > 0.004) our computation is fully 
numerical. Since we determine critical lines by studying the susceptibility, the transition is 
continuous. In other words, a possibility of a first order transition is not considered in the 
present analysis. 

A. Results for t' = 

Figure [2] shows the phase diagram for i! = 0. As mentioned in Sec. II. B, five different 
types of charge instability are found: <iCDW, dPI, BOP x , BOP xy , and PS. The instability 
toward the commensurate dCDW, namely with q = (tt,tv), occurs in a wide doping region. 
The transition temperature decreases gradually with increasing hole density and exhibits 
reentrant behavior at low T in the region 0.12 < 5 < 0.14. However, near 5 ~ 0.14 an 
incommensurate [q ^ (tt, 7r)] rfCDW instability occurs below T « 0.015 and its critical 
doping rate is higher than that of the commensurate dCDW. Hence the resulting critical 
line of the dCDW follows the outer line, i.e., the thin line at low T and the thick line for 
high T. 

For t' = the commensurate <iPI with q = (0,0) has the same onset temperature as the 
commensurate BOP x and BOP xy , namely with q = (tt,tt); this reason will be explained in 
the last paragraph in the present section. The transition lines exhibit reentrant behavior at 
low T. However, an incommensurate <iPI emerges at low T and preempts the reentrant line, 
extending the region of dPI. BOP^ and P>OP xy also exhibit an incommensurate instability at 
low T and preempt their commensurate instabilities. Furthermore the degeneracy between 
BOP x and BOP^ is lifted via an incommensurate transition. While the critical doping rate 
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E-,0.02 




FIG. 2: (Color online) Critical temperature versus doping rate for dCDW, dPI, BOP x , BOP xy , and 
PS for t 1 = and J = 0.3. Thick (thin) lines describe commensurate (incommensurate) transitions. 
The critical line for PS is shown in a larger scale of T in the inset. 

for BOP becomes higher than for dPI at low T, this result occurs only for a small t' and, as 
will be shown below, the opposite occurs in the presence of a realistic t' for cuprates. 

The system also exhibits PS at low doping. The inset of Fig. 2 shows the PS line in a 
larger temperature scale. We see that PS appears with decreasing temperature, but with 
further cooling down it goes back to the paramagnetic phase. As a result, PS occurs only 
in an intermediate temperature region. This peculiar reentrant behavior was also found in 



Ref. [74] in the Hubbard model. The region of PS shrinks with increasing V and also by 
introducing £'(< 0), as will be discussed in the subsection D. 

The phase diagram in Fig. [2] should not be interpreted in such a way that the ciCDW is 
unstable against the dPI or BOP at low T or low 5, because we perform a stability analysis 
in the paramagnetic phase. Rather, Fig. [^indicates a hierarchy for different charge instabil- 
ities, i.e., the outer the critical line is, the stronger the tendency toward the corresponding 
instability is. 

It requires highly accurate numerics to determine precisely a modulation vector q of 
each order along its outer critical line because of a rather flat structure of the susceptibility 
in momentum space, especially for the <iPI. Therefore considering our achieved numerical 
accuracy we present in Fig. [3] modulation vectors of each instability, for which the absolute 
value of the corresponding eigenvalue of D~^(q, 0) becomes less than 10~H on its outer 
critical line. The width of such a q region, at a fixed temperature, implies how sharp 
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the susceptibility is in momentum space. Since modulation vectors of each instability are 
computed along its outer critical line, each critical temperature shown in Fig. [3] corresponds 
to a certain critical doping rate, which can be read off from Fig. |5J Although Fig. [3] is 
presented only along the axis (tt, 0)-(ir, 7r)-(0, 0)-(7r, 0), we scanned the whole q region of the 
Brillouin zone and checked numerically that instabilities indeed occur along that axis. 
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_J dPI 






BOPx 



(71,0) (71,71) q (0,0) (71,0) (7C,7C) q (0,0) (71,0) (71,71) q (0,0) (71,0) 



FIG. 3: (Color online) Modulation vectors for dCDW (left panel), dPI (middle panel), and BOP^ 
(right panel) along the corresponding outer critical line in Fig. [2j For each critical temperature, 
the critical doping rate can be read off from Fig. [2j Because of symmetry, the results along the 
direction of (0, tt)-(tt, tt) are the same as those along the (tt, 0)-(tt, tt) direction for the dCDW and 
dPI. 

In Fig. [3] (a) we show the result for the ciCDW along its outer critical line in Fig. [2j At 
high critical temperature (i.e., low critical doping rate) the instability occurs at q = (7r,7r), 
and with lowering temperature the modulation vector shifts from (7r, tt) and becomes incom- 
mensurate. In Fig. |3](c) we plot the corresponding modulation vector of the BOP x . At low 
T, the modulation vector q slightly shifts from (tt, tt) and the BOP^ becomes incommensu- 
rate, as in the case of the <iCDW. At high T, q is located at (tt, tt), but in contrast to the 
case of the ciCDW, the q region is not extended on the side of the direction of (tt, tt)-(0, 0). 
This is because the eigenvector of the BOPa, does not exist there, instead, the eigenvector 
of the full susceptibility [Eq. ( fl6l) ] changes to that corresponding to the dPI. A modulation 
vector of the BOP^ appears only along the axis (tt, 7r)-(0, 0). Its T c dependence is very 
similar to that of BOP x , but the labels (tt, 0) and (0, 0) in Fig. [3] (c) should be replaced by 
(0,0) and (tt,0), respectively. 

The corresponding result for the <iPI is shown in Fig. [3] (b), which looks very different 
from those for the dCDW and BOP. In fact, for t' = 0, the static electronic polarizability of 
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the dPl has a special feature, which was already noted in Ref. 
see this we rewrite Eq. (J23j) in a different form, 
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in a different context. To 



rLjpi(q,0) = ~J] n F (e k ) 



N s 
s k 



7dPi(q, k ) 7dPi(-q, k ) 



. Ck _ e k+q Ck — ^k-q 

When q lies along the diagonal direction q || (q, q), we find after some algebra 



(32) 



tt / n\ / \ + k y k x ky . k x k y 

ndPl(q ' 0) = (t5 + 2A)N a ^ " F(ek) C ° S Sm ' (33) 

that is, the static dPl susceptibility [Eq. fl23|) ] does not depend on q for any momentum 
along the diagonal direction. This result holds for any carrier density and any temperature. 
Therefore, if the <iPI takes place for a vector q in the diagonal direction, the susceptibility 
diverges simultaneously at all q along the diagonal direction. The full susceptibility [Eq. ffTB"]) ] 
actually shows that feature in Fig. [3] (b). Furthermore, this flat feature of the susceptibility 
extends more away from the diagonal direction. The q region along (0, 0)-(tt, 0) shrinks 
at T c Rj 0.024 in Fig. [3] (b), which results from the proximity to PS, as will be discussed 
in the subsection C. While the susceptibility is always flat along the diagonal direction 
of q, the susceptibility shows a peak at a modulation vector along (0, 0)-(tt, 0) at low T. 
The q region has a sharp boundary at (it, it) in Fig. [3] (b) and the <iPI does not have any 
possible modulation vector along the (tt, 0)-(tt, tt) direction. This is because the eigenvector 
corresponding to the dPl is not realized along (tt, 0)-(7r, tt), instead, the BOP x eigenvector 
appears there. This property may be understood also in terms of the effective susceptibilities. 
Equation (1291) indicates that if X^i(q')O) becomes zero, either Xbop^ (q'j 0) or XboPj, (i'j 0) 
should be already negative, since in general Xbop^ is not equal to Xbop^ for a momentum 
along (tt, 0)-(tv, tt), except for q' = (tt, tt) where both Xbop^*!'' 0) anc ^ XBOPyi^' >®) can 
become zero simultaneously. Therefore a possible instability of the <iPI along (tt, 0)-(tt, tt) is 
replaced by the BOP^. 

The q-independence of Hdpi along the diagonal direction leads to another special feature. 
As we mentioned at the end of Sec. II, the onset temperature of the BOP with q = (tt, tt) 
is the same as that of the c£PI with q = (tt, tt) [Eqs. fl30|) and fl3T]) ]. Therefore the onset 
temperature of the commensurate BOP becomes the same as that of the dPl with q = 0, 
as shown in Fig. [2j 
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FIG. 4: (Color online) Critical temperature and doping rate for if = —0.20 (upper panel) and 
t' = —0.30 (lower panel). The notation is the same as Fig. [2j 

B. Results for finite t' 

The degeneracy between the dPl and BOP seen in Fig. [2] is lifted by introducing t'. The 
upper and lower panels in Fig. H] show the results for if = —0.20 and —0.30, respectively. 
While the BOP instability is always restricted to a lower doping region, the <iPI becomes 
favorable in a wider doping region. Near half-filling the <iPI and BOP are still almost 
degenerate because, as seen from Eq. (fT2l . the hopping integral t' is renormalized to be 
t'r oc if 5 and becomes irrelevant close to half-filling. The BOP x and BOP xy are always 
degenerate as far as they exhibit a commensurate instability. Their degeneracy is lifted 
when their modulation vector becomes incommensurate at low temperature. 

As shown in Fig. HI the doping region of the commensurate <iCDW instability is extended 
by the presence of if and an incommensurate dCDW becomes dominant at high 5 and low 
critical temperature. On the other hand, PS is suppressed by introducing if . The critical 
line for PS bends back to zero doping for high T (not shown) in a way similar to the case 
for H = (inset of Fig. E}. 

The modulation vector of each instability is shown in the upper and middle row in 
Fig. [5] for t' = —0.20 and —0.30, respectively, along the corresponding outer critical line in 
Fig. H Both dCDW [Figs. E (a) and (d)] and BOP^ [Figs. M (c) and (f )] show an instability 
at q = (jr, w) for high critical temperature, and becomes incommensurate for low critical 
temperature. These features are the same as those seen in Figs. [3] (a) and (c). Results of 
BOP xy are essentially the same as those of BOP^, but the labels (ir, 0) and (0, 0) should be 
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FIG. 5: (Color online) Modulation vectors for dCDW (left panels), dPI (middle panels), and BOP^ 
(right panels) for t' = —0.20 (upper row), if = —0.30 (middle row), and t' = —0.40 (lower row) 
along the outer critical line of the corresponding order for each t' in Fig. 0] and Fig. [6] (middle) . 

interchanged in Figs. |5] (c) and (f). 

Figs. [5] (b) and (e) show results for the dPI. They are very different from Fig. [3] (b), 
except for a region of high critical temperature near T c « 0.03, namely the doping region 
< 5 C < 0.02, where the effect of t' becomes irrelevant. For t' = —0.20 [Fig. [5] (b)], as 
the critical temperature decreases, the q region shrinks around q = (0, 0) and the g?PI 
tends to become commensurate. Close to zero temperature, however, a tendency toward an 
incommensurate dPI appears in the (0,0)-(7r, 0) direction. This incommensurate feature is 
also seen more clearly for t' — at low T [Fig. [3] (b)], and disappears quickly with increasing 
if . It becomes nearly invisible for if = —0.30 in our temperature scale. For if = —0.30, 
the q region shrinks first around q = (0, 0) with decreasing the critical temperature. In the 
intermediate temperature range, 0.021 > T c > 0.008, the modulation vector becomes slightly 
incommensurate along the diagonal. This deviation from the commensurate vector is also 
barely visible in the bottom panel of Fig. HI where the incommensurate <iPI line separates 
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FIG. 6: (Color online) The same plot as Fig. [H but for different choices of t': if = —0.35 (top 
panel), if = —0.40 (middle panel), and if = —0.45 (bottom panel). In the latter two cases the 
phase diagram in a high doping region is magnified in the inset. 

very slightly from the commensurate dPl in the corresponding temperature region. For 
T c < 0.008, the dPI becomes fully commensurate. 

The phase diagram close to half- filling does not depend essentially on a choice of if. In 
fact, the critical lines for the BOP and PS do not change much even for a further larger 
if. However, we find that tendencies toward ciCDW and dPl have strong if dependence. In 
Fig. |6]we present the phase diagram for if = —0.35, —0.40, and —0.45. First we focus on 
the c?CDW for if = —0.35 and —0.40. The instability extends to a higher doping region with 
increasing if and the commensurate dCDW tends to become more favorable even at low T. 
The doping region of the rfCDW, however, starts to decrease quickly for \t'\ > 0.41, as can 
be seen in the result for if = —0.45. 

In contrast to the dCDW, the outer critical line for the dPl extends to higher doping 
with increasing t'. For t' = —0.35, an incommensurate dPI becomes dominant in a wide 
doping region (0.05 < 5 < 0.25). While the commensurate dPl is realized for 5 < 0.16 
and 5 > 0.19, it does not occurs between these two doping region. This feature is more 
evident for t' = —0.40 [Fig. [6] (middle panel)]. In a wide doping region (0.10 < 5 < 0.30) the 
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commensurate dPI does not occur and only an incommensurate dPI is possible. However 
in a high doping region (0.30 < 5 < 0.35), the commensurate dPI shows up again; see the 
inset in Fig. (middle panel). These peculiar features of the dPI are due to the presence 
of the van Hove singularity around 5 = 0.33, where the d-wave weighted density of states,— 
which is defined by lim^o riipi(q, 0) in Eq. f[2"4"|) . is enhanced, favoring the instability at 
q = (0, 0). In an intermediate doping region, the <i-wave density of states is suppressed and 
the dPI with a finite q becomes more favorable. Closer to half-filling, however, the ci-wave 
density of states is again enhanced because of narrowing the band width upon approaching 
half-filling, leading to a recovery of the commensurate dPI for 5 < 0.10. For t' = —0.45 
these features are more emphasized. Figure O (bottom panel) shows that the commensurate 
dPI occurs both, close to half-filling and around van Hove filling (5 = 0.42), and these two 
regions are connected by the critical line of an incommensurate dPI. The commensurate <iPI 
close to half-filling and in a high doping region was also found in the slave-boson mean-field 
theory.— 

In Fig. [5]we compare the modulation vector of each instability for t' = —0.20, —0.30, and 
—0.40. With increasing t' , the <iCDW tends to be more commensurate even at low critical 
temperature. For the dPI, on the other hand, a large t' tends to favor an incommensurate 
modulation along the diagonal direction of the Brillouin zone in an intermediate range of 
a critical temperature, and the commensurate dPI can be realized only at high and low 
critical temperature, corresponding to a doping region close to half-filling and around van 
Hove filling, respectively. The modulation vectors of the BOP x (and also BOP xy ) do not 
depend much on a choice of i! . 

C. Mutual interaction among different modes 

Because of the renormalization of the bosonic propagators due to the coupling to elec- 
tronic bubbles [Eq. ffTB"]) ]. one would expect in general some coupling among different modes. 
This effect actually appears for modulation vectors of the cfPI. There is a dip on the side of 
the region of (0, 0)-(tt, 0) at T c w 0.024 in Fig. [3] (b), and T c « 0.025 and 0.026 in Figs. E (b) 
and (e), respectively. This dip occurs near the temperature where the critical lines of the <iPI 
and PS cross each other (see Figs. [2] and H]). Moreover, we checked that the dip in question 
does not appear in results obtained from the effective susceptibility of the dPI [Eq. (I2"3"j) ]. 



20 



Therefore a tendency toward PS plays a role in the suppression of the incommensurate 
feature of the dPl along (0, 0)-(n, 0). 

Except for the above feature, we checked that our results (Figs. [2J1HD are nearly the same 
as those determined by the effective susceptibilities [Eqs. f[2"0"j) . (j2Tj) . fT2"3"j) . and (J23|) ]. In this 
sense, the coupling among different bosonic fluctuations is rather weak at least in leading 
order. 

D. Effect of the V-term and stability of phase separation 

We checked that the results for <iCDW, dPl, and BOP are almost intact for different 
choices of V (> 0) and that an additional instability such as the usual checkerboard charge 
density wave does not occur at least for V < 1 for any doping rate.— ^ Furthermore, the 
reentrant critical line of PS (inset of Fig. |2]) is a robust feature. However, it is a subtle issue 
whether PS actually occurs at T = 0. The result depends on choices of V, t', and J. For 
if = and J = 0.3, we found that PS at T = occurs for 5 < 0.08 (0.025) at V = (0.1) 
and vanishes already for V > 0.2. When i! is introduced, PS is strongly suppressed even at 
V = 0, for example, it occurs at T = only for 5 < 0.01 for i! = —0.35. A smaller J also 
suppresses PS, and for the special case of J = no PS is observed at T = for any V (> 0) 
and i! (< 0). 

IV. CONCLUSION 

Applying a large- iV expansion formulated in the path integral representation of the t-J 
model, we have analyzed all possible charge instabilities of the paramagnetic phase, and 
have elucidated the phase diagram in the doping and temperature plane for a sequence of t' . 
We have found that <iCDW, dPl, BOP, and PS are the most important charge instabilities. 
The first two instabilities appear in a wide doping region. The <iCDW usually becomes 
the leading instability and the <iPI occurs as a next leading one with a strong tendency to 
become incommensurate. In the presence of a large t', however, we have found that the <iPI 
becomes the leading instability in a high doping region. Considering the high complexity of 
the t-J model, it is beyond the scope of the present study to address which charge instability 
would become ultimately the leading one. Rather, the present stability analysis on charge 
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instabilities of the paramagnetic state was motivated by active studies on the PG in cuprates 
in terms of various charge fluctuations, and we have provided a microscopic basis of possible 
charge fluctuations in doped Mott insulators. We first compare our results with literature 
and then discuss implications for cuprate superconductors. 

Taking into account a number of papers about charge stripes in cuprates,— ^ it may be 
surprising that we do not detect stripe tendencies in our formalism, which exclusively favors 
charge instabilities. If the t-J model would exhibit a tendency to charge stripes order, it 
might be a consequence of a coupling with incommensurate magnetic modes, which could 
appear in the next-to-leading order [0(1/N)] in the present scheme. A charge-stripe solution 
was actually obtained in the t-J model with t' = in the DMRG study,— which however 
contradicts other studies.—^ In the presence of t', on the other hand, most of numerical 
studies in the t-J model reported that the charge stripe solution becomes unstable.—^ Our 
results, therefore, agree to major literature. 

Interestingly, the BOP x ^ shares the same feature as stripe order from a symmetry point 
of view. When q shifts away from (71*, 7r), q is located only along the q y M direction and thus 
the BOP x (j / ) breaks both orientational and translational symmetry of the lattice. In fact such 
an incommensurate BOP^) instability is found to occur up to 5 ~ 0.10; see Figs. [21 HJ and 
[6j While the BOP has not been discussed much so far, the BOP was also obtained in other 
studies in the t-J model.— ~— 

As discussed in Sec. III.D, PS at T = strongly depends on a choice of V, t', and J. 
However it is a robust feature that PS occurs in an intermediate temperature region. This 
property for a finite T has not been discussed so far except for Ref. |74| in the Hubbard model, 
probably because various numerical simulations are usually coded only at zero temperature. 
Interestingly, the reentrant critical line of PS (see the inset of Fig. [2]) was interpreted as a 
source to generate a strong forward scattering channel of the electron-phonon vertex which 
emerges only at finite T.— 

While the commensurate <iPI (q = 0) was already found in the t-J— and Hubbard^ 
models, an incommensurate <iPI (q 7^ 0) started to be discussed quite recently.— ~— We have 
obtained that the static <i-wave electronic polarizability [Eq. fl33|) ] does not depend on q for 
any momentum along the Brillouin zone diagonal, which holds for any temperature and any 
electron density as long as t! is zero. In our model, this feature remains even for a finite 
t' near half-filling since t' is effectively reduced by a factor of 5. Our result shown in the 
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inset of Fig. [6] (bottom panel) may be best compared with existing results, since they were 
obtained in a weak coupling analysis.— ~— At hole density below van Hove filling (5 < 0.42), 
the leading instability is an incommensurate dPl and its modulation vector is located along 
the (0, 0)-(7r, 7r) direction, which agrees with literature.— ~— At hole density above van Hove 
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showed that the static electronic 



filling we have obtained the dPl with q = 0, while Ref. 
polarizability of the dPl has a peak along the (0,0)-(7r, 0) direction at least at T = 0. In 
our case, an incommensurate peak along the (0, 0)-(7r, 0) direction indeed develops as seen 
in Figs. [3] (b), |5] (b) for a small t', but it develops below extremely low temperature for a 
large t'. This effect is not visible in Fig. E] (bottom). 

In the 1/N expansion, the <iCDW is the leading instability in most of cases, in agree- 
ment with previous studies.— ~— We have found that close to the c/CDW, the <iPI also exists 
in a wide doping region. Therefore fluctuations associated with both rfCDW and dPl are 
expected to be important for temperatures above the onset of the <iCDW. The mutual inter- 
action between c/CDW and <iPI seems rather weak at least in leading order because both, the 
full calculation [Eq. (TT6])] and effective susceptibilities [Eqs. (ED}, $23$, (E3]), and (125]) ]. give 
nearly the same results. Given that the presence of t' is usually assumed for cuprates and our 
critical lines of rfCDW and <iPI have the same doping dependence of the PG temperature, 
furthermore with a comparable temperature if t ~ 500 meV, it is interesting to study each 
fluctuation effect on the electronic property. In fact, existing work already showed interesting 
results, but with some open questions. The phenomenological study assuming the dCDW 
showed that essential features of the PG are well captured™' However fluctuation effects 



were not considered in Ref. 
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In a perturbative calculation of the electron self-energy due 
to a coupling to dCDW fluctuations in the t-J model, a pseudogap is obtained in the elec- 
tronic spectral function, which shares many important features with experimental data.— ~— 
Moreover the same analysis was also applied to the explanation of anisotropic scattering rate 
of quasi-particles 7 - observed in angle-dependent magnetoresistance experiments.— However, 
no calculation was performed beyond the perturbative analysis. On the other hand, in a 
perturbative calculation for <iPI fluctuations centered around q = (0,0), a splitting of the 
spectral function near the Fermi energy was obtained, reminiscent of a pseudogap.— Going 
beyond the perturbation theory and summing up all diagrams in the Gaussian fluctuation 
regime, however, instead of a splitting, the spectral function exhibits a broad single peak 
centered at the Fermi energy with a strong k dependence of ci-wave symmetry— The spec- 
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trum in the Ginzburg region^ 2 - is an open question. Furthermore, the role of incommensurate 
dPl fluctuations on a pseudogap phenomenon remains elusive. 

The dPl couples directly with xy anisotropy such as due to a lattice structure and an exter- 
nal strain. While the dPl changes to a crossover phenomenon in such a case, the anisotropy 
can be strongly enhanced by the underlying dPl fluctuations as already discussed.— 1 ^ The 
same idea is also discussed for iron-pnictide superconductors near the structural phase tran- 
sition from the tetragonal to orthorhombic phase.— Given that lattice anisotropy frequently 
exists in cuprates, the relevance of the dPl channel in the t-J model suggests important im- 
plications for understanding cuprate superconductors, not only for Y-basedii^— but also 
for La-based 3 ^ 1 ^— compounds. 

For a large t', the commensurate dPl appears in a heavily overdoped region around van 
Hove filling (5 ~ 0.3 — 0.45 in Fig. [6]). While our critical line exhibits reentrant behavior at 
low T, the canonical model for the <iPI— ^ suggests that the reentrant behavior is preempted 
by a first order transition as a function of the chemical potential, or equivalently a phase 
separation as a function of doping, as far as the dPl occurs at q = 0. It is known that 
Sr3Ru207 exhibits the dPl in a magnetic field.— _ — In addition, a highly overdoped region in 
cuprates, where no superconducting and antiferromagnetic instabilities are expected, may 
also provide an opportunity to explore the dPl. 
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